PalQuant: Accelerating High-Precision Networks on Low-Precision Accelerators

AbstractRecently low-precision deep learning accelerators (DLAs) have become popular due to their advantages in chip area and energy consumption, yet the quantized models on these DLAs bring severe accuracy degradation. One way achieve both high efficient inference is deploy high-precision neural networks DLAs, which rarely studied. In this paper, we propose PArallel Low-precision Quantization (PalQuant) method that approximates computations via parallel representations from scratch. addition, present a novel cyclic shuffle module boost cross-group information communication between groups. Extensive experiments demonstrate PalQuant has superior performance state-of-the-art quantization methods speed, e.g., for ResNet-18 network quantization, can obtain 0.52% higher 1.78\(\times \) speedup simultaneously over 4-bit counter-part 2-bit accelerator. Code available at https://github.com/huqinghao/PalQuant.KeywordsQuantizationNetwork accelerationCNNs